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Abstract 

This  paper  considers  a  class  of  strategic  scenarios  in  which  two  networks  of  agents  have  opposing  objectives  with  regards  to 
the  optimization  of  a  common  objective  function.  In  the  resulting  zero-sum  game,  individual  agents  collaborate  with  neighbors 
in  their  respective  network  and  have  only  partial  knowledge  of  the  state  of  the  agents  in  the  other  network.  For  the  case 
when  the  interaction  topology  of  each  network  is  undirected,  we  synthesize  a  distributed  saddle-point  strategy  and  establish 
its  convergence  to  the  Nash  equilibrium  for  the  class  of  strictly  concave-convex  and  locally  Lipschitz  objective  functions.  We 
also  show  that  this  dynamics  does  not  converge  in  general  if  the  topologies  are  directed.  This  justifies  the  introduction,  in  the 
directed  case,  of  a  generalization  of  this  distributed  dynamics  which  we  show  converges  to  the  Nash  equilibrium  for  the  class 
of  strictly  concave-convex  differentiable  functions  with  globally  Lipschitz  gradients.  The  technical  approach  combines  tools 
from  algebraic  graph  theory,  nonsmooth  analysis,  set- valued  dynamical  systems,  and  game  theory. 
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1  Introduction 

Recent  years  have  seen  an  increasing  interest  on  net¬ 
worked  strategic  scenarios  where  agents  may  cooperate 
or  compete  with  each  other  towards  the  achievement  of 
some  objective,  interact  across  different  layers,  have  ac¬ 
cess  to  limited  information,  and  are  subject  to  evolving 
interaction  topologies.  This  paper  is  a  contribution  to 
this  body  of  work.  Specifically,  we  consider  a  class  of 
strategic  scenarios  in  which  two  networks  of  agents  are 
involved  in  a  zero-sum  game.  We  assume  that  the  objec¬ 
tive  function  can  be  decomposed  as  a  sum  of  concave- 
convex  functions  and  that  the  networks  have  opposing 
objectives  regarding  its  optimization.  Agents  collaborate 
with  the  neighbors  in  their  own  network  and  have  partial 
information  about  the  state  of  the  agents  in  the  other 
network.  Such  scenarios  are  challenging  because  infor¬ 
mation  is  spread  across  the  agents  and  possibly  multiple 
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layers,  and  networks,  by  themselves,  are  not  the  decision 
makers.  Our  aim  is  to  design  a  distributed  coordination 
algorithm  that  can  be  used  by  the  agents  to  converge 
to  the  Nash  equilibrium.  Note  that,  for  a  2-player  zero- 
sum  game  of  the  type  considered  here,  a  pure  Nash  equi¬ 
librium  corresponds  to  a  saddle  point  of  the  objective 
function. 

Literature  review.  Multiple  scenarios  involving  net¬ 
worked  systems  and  intelligent  adversaries  in  sensor 
networks,  filtering,  finance,  and  wireless  communica¬ 
tions  (Kim  and  Boyd,  2008;  Wan  and  Lemmon,  2009) 
can  be  cast  into  the  strategic  framework  described 
above.  In  such  scenarios,  the  network  objective  arises 
as  a  result  of  the  aggregation  of  agent-to-agent  ad¬ 
versarial  interactions  regarding  a  common  goal,  and 
information  is  naturally  distributed  among  the  agents. 
The  present  work  has  connections  with  the  literature  on 
distributed  optimization  and  zero-sum  games.  The  dis¬ 
tributed  optimization  of  a  sum  of  convex  functions  has 
been  intensively  studied  in  recent  years,  see  e.g.  (Nedic 
and  Ozdaglar,  2009;  Wan  and  Lemmon,  2009;  Johans¬ 
son  et  ah,  2009;  Zhu  and  Martinez,  2012).  These  works 
build  on  consensus-based  dynamics  (Olfati-Saber  et  al., 
2007;  Ren  and  Beard,  2008;  Bullo  et  al.,  2009;  Mes- 
bahi  and  Egerstedt,  2010)  to  find  the  solutions  of  the 
optimization  problem  in  a  variety  of  scenarios  and  are 
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designed  in  discrete  time.  Exceptions  include  (Wang 
and  Elia,  2010,  2011)  on  continuous-time  distributed 
optimization  on  undirected  networks  and  (Gharesifard 
and  Cortes,  2012c)  on  directed  networks. 

Regarding  zero-sum  games,  the  works  (Arrow  et  al., 
1958;  Maistroskii,  1977;  Nedic  and  Ozdgalar,  2009) 
study  the  convergence  of  discrete-time  subgradient  dy¬ 
namics  to  a  saddle  point.  Continuous-time  best-response 
dynamics  for  zero-sum  games  converges  to  the  set  of 
Nash  equilibria  for  both  convex-concave  (Hofbauer  and 
Sorin,  2006)  and  quasiconvex-quasiconcave  (Barron 
et  ah,  2010)  functions.  Under  strict  convexity-concavity 
assumptions,  continuous-time  subgradient  flow  dynam¬ 
ics  converges  to  a  saddle  point  (Arrow  et  ah,  1951, 
1958).  Asymptotic  convergence  is  also  guaranteed  when 
the  Hessian  of  the  objective  function  is  positive  defi¬ 
nite  in  one  argument  and  the  function  is  linear  in  the 
other  (Arrow  et  ah,  1958;  Feijer  and  Paganini,  2010). 
The  saddle-point  and  learning  dynamics  reviewed  above 
are  not  directly  applicable  to  the  problem  considered 
here  because  of  the  information  about  the  game  is  dis¬ 
tributed  across  the  networked  scenario  and  is  not  cen¬ 
trally  available  anywhere.  The  distributed  computation 
of  Nash  equilibria  in  noncooperative  games,  where  all 
players  are  adversarial,  is  a  challenging  problem.  The 
algorithm  in  (Li  and  Ba§ar,  1987)  relies  on  all-to-all 
communication  and  does  not  require  players  to  know 
each  other’s  payoff  functions  (which  must  be  strongly 
convex).  In  (Frihauf  et  al.,  2012;  Stankovic  et  ah,  2012), 
players  are  unaware  of  their  own  payoff  functions  but 
have  access  to  the  payoff  value  of  an  action  once  it  has 
been  executed.  These  works  design  distributed  strate¬ 
gies  based  on  extremum  seeking  techniques  to  seek  the 
set  of  Nash  equilibria. 

Statement  of  contributions.  We  introduce  the  prob¬ 
lem  of  distributed  convergence  to  Nash  equilibria  for 
two  networks  engaged  in  a  strategic  scenario.  The  net¬ 
works  aim  to  either  maximize  or  minimize  a  common 
objective  function  which  can  be  written  as  a  sum  of 
concave-convex  functions.  Individual  agents  collaborate 
with  neighbors  in  their  respective  network  and  have 
partial  knowledge  of  the  state  of  the  agents  in  the  other 
one.  Our  first  contribution  is  the  introduction  of  an 
aggregate  objective  function  for  each  network  which 
depends  on  the  interaction  topology  through  its  Lapla- 
cian  and  the  characterization  of  a  family  of  points  with 
a  saddle  property  for  the  pair  of  functions.  We  show 
the  correspondence  between  these  points  and  the  Nash 
equilibria  of  the  overall  game.  When  the  graphs  describ¬ 
ing  the  interaction  topologies  within  each  network  are 
undirected,  the  gradients  of  these  aggregate  objective 
functions  are  distributed.  Building  on  this  observation, 
our  second  contribution  is  the  synthesis  of  a  consensus- 
based  saddle-point  strategy  for  adversarial  networks 
with  undirected  topologies.  We  show  that  the  proposed 
dynamics  is  guaranteed  to  asymptotically  converge  to 
the  Nash  equilibrium  for  the  class  of  strictly  concave- 
convex  and  locally  Lipschitz  objective  functions.  Our 


third  contribution  focuses  on  the  directed  case.  We  show 
that  the  transcription  of  the  saddle-point  dynamics  to 
directed  topologies  fails  to  converge  in  general.  This 
leads  us  to  propose  a  generalization  of  the  dynamics, 
for  strongly  connected  weight-balanced  topologies,  that 
incorporates  a  design  parameter.  We  show  that,  by  ap¬ 
propriately  choosing  this  parameter,  the  new  dynamics 
asymptotically  converges  to  the  Nash  equilibrium  for 
the  class  of  strictly  concave-convex  differentiable  ob¬ 
jective  functions  with  globally  Lipschitz  gradients.  The 
technical  approach  employs  notions  and  results  from 
algebraic  graph  theory,  nonsmooth  and  convex  anal¬ 
ysis,  set-valued  dynamical  systems,  and  game  theory. 
As  an  intermediate  result  in  our  proof  strategy  for  the 
directed  case,  we  provide  a  generalization  of  the  known 
characterization  of  cocoercivity  of  concave  functions  to 
concave-convex  functions. 

The  results  of  this  paper  can  be  understood  as  a  gen¬ 
eralization  to  competing  networks  of  the  results  we  ob¬ 
tained  in  (Gharesifard  and  Cortes,  2012c)  for  distributed 
optimization.  This  generalization  is  nontrivial  because 
the  payoff  functions  associated  to  individual  agents  now 
also  depend  on  information  obtained  from  the  opposing 
network.  This  feature  gives  rise  to  a  hierarchy  of  saddle- 
point  dynamics  whose  analysis  is  technically  challeng¬ 
ing  and  requires,  among  other  things,  a  reformulation  of 
the  problem  as  a  constrained  zero-sum  game,  a  careful 
understanding  of  the  coupling  between  the  dynamics  of 
both  networks,  and  the  generalization  of  the  notion  of 
cocoercivity  to  concave-convex  functions. 

Organization.  Section  2  contains  preliminaries  on  non¬ 
smooth  analysis,  set-valued  dynamical  systems,  graph 
theory,  and  game  theory.  In  Section  3,  we  introduce  the 
zero-sum  game  for  two  adversarial  networks  involved 
in  a  strategic  scenario  and  introduce  two  novel  aggre¬ 
gate  objective  functions.  Section  4  presents  our  algo¬ 
rithm  design  and  analysis  for  distributed  convergence  to 
Nash  equilibrium  when  the  network  topologies  are  undi¬ 
rected.  Section  5  presents  our  treatment  for  the  directed 
case.  Section  6  gathers  our  conclusions  and  ideas  for  fu¬ 
ture  work.  Appendix  A  contains  the  generalization  to 
concave-convex  functions  of  the  characterization  of  co¬ 
coercivity  of  concave  functions. 

2  Preliminaries 

We  start  with  some  notational  conventions.  Let  R,  R>o, 
Z,  Z>i  denote  the  set  of  real,  nonnegative  real,  integer, 
and  positive  integer  numbers,  respectively.  We  denote 
by  ||  •  ||  the  Euclidean  norm  on  Rd,  d  £  Z>i  and  also 
use  the  short-hand  notation  1^  =  (1, . . . ,  1)T  and  0<j  = 
(0, . . .  ,0)r  £  Rd.  We  let  1^  denote  the  identity  matrix 
in  Rdxd.  For  matrices  A  £  RdlXd2  ancj  g  g  ReiXe2, 
di,d2,ei,e2  £  Z> i ,  we  let  A ®  B  denote  their  Kronecker 
product.  The  function  /  :  X i  x  X2  — »  M,  with  Xi  C  Rrfl, 
X2  C  Rd2  closed  and  convex,  is  concave-convex  if  it  is 
concave  in  its  first  argument  and  convex  in  the  second 
one  (Rockafellar,  1997).  A  point  {x\,x^)  £  Xi  x  X2  is 
a  saddle  point  of  /  if  f(xi,  x^)  <  /(#*,  xlf)  <  /(a;*,  x-f) 
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for  all  X\  £  Xi  and  X2  £  X2.  Finally,  a  set-valued  map 
/  :  Rd  =4  Rd  takes  elements  of  Rd  to  subsets  of  Rd. 

2.1  Nonsmooth  analysis 

We  recall  some  notions  from  nonsmooth  analy¬ 
sis  (Clarke,  1983).  A  function  /  :  Rd  — >  R  is  locally 
Lipschitz  at  x  £  Rd  if  there  exists  a  neighborhood  U  of 
x  and  Cx  £  R>0  such  that  | f(y)  -  f(z)\  <  Cx\\y  -  z\\, 
for  y,  z  £  U.  f  is  locally  Lipschitz  on  Rd  if  it  is  locally 
Lipschitz  at  x  for  all  x  £  Rd  and  globally  Lipschitz  on 
Rd  if  for  all  y,  z  £  Rd  there  exists  C  £  R>o  such  that 
| f(y)  —  f(z) |  <  C\\y  —  z\\.  Locally  Lipschitz  functions 
are  differentiable  almost  everywhere.  The  generalized 
gradient  of  /  is 

df(x)  =  co<  lim  V/(xfc)  |  xk  ->  x,  xk  £  Llf  U  S \, 

l  k—> 00  ) 

where  17/  is  the  set  of  points  where  /  fails  to  be  differ¬ 
entiable  and  S  is  any  set  of  measure  zero. 

Lemma  2.1  (Continuity  of  the  generalized  gradi¬ 
ent  map):  Let  f  :  Rd  — >  R  be  a  locally  Lipschitz  func¬ 
tion  at  x  £  Rd.  Then  the  set-valued  map  df  :  Rd  =3  Rd 
is  upper  semicontinuous  and  locally  bounded  at  x  £  Rd 
and  moreover,  df{x)  is  nonempty,  compact,  and  convex. 
For  /  :  Rd  x  Rd  — >  R  and  z  £  Rd,  we  let  dxf(x,  z)  de¬ 
note  the  generalized  gradient  of  x  ^  f(x,  z).  Similarly, 
for  x  £  Rd,  we  let  dzf{x,z)  denote  the  generalized  gra¬ 
dient  of  2£  f{x,  z).  A  point  x  £  Rd  with  0  £  df(x)  is 
a  critical  point  of  /.  A  function  /  :  Rd  — >  R  is  regular  at 
x  £  R  if  for  all  v  £  Rd  the  right  directional  derivative  of 
/,  in  the  direction  of  v,  exists  at  x  and  coincides  with  the 
generalized  directional  derivative  of  /  at  x  in  the  direc¬ 
tion  of  v.  We  refer  the  reader  to  (Clarke,  1983)  for  defi¬ 
nitions  of  these  notions.  A  convex  and  locally  Lipschitz 
function  at  x  is  regular  (Clarke,  1983,  Proposition  2.3.6). 
The  notion  of  regularity  plays  an  important  role  when 
considering  sums  of  Lipschitz  functions. 

Lemma  2.2  (Finite  sum  of  locally  Lipschitz  func¬ 
tions):  Let  {/*}”=i  be  locally  Lipschitz  at  x  £  Rd.  Then 
<9(E"=i  P)(x)  C  E"=  1dfi(x),  and  equality  holds  if  f* 
is  regular  fori  £  {1, . . .  ,n}. 

A  locally  Lipschitz  and  convex  function  /  satisfies,  for 
all  x,  x'  £  Rd  and  £  £  8f(x),  the  first-order  condition  of 
convexity, 

f(x')  -  f{x)  >  £  ■  {x'  -  x).  (1) 

2.2  Set-valued  dynamical  systems 

Here,  we  recall  some  background  on  set-valued  dynami¬ 
cal  systems  following  Cortes  (2008).  A  continuous-time 
set-valued  dynamical  system  on  X  C  Rd  is  a  differential 
inclusion 

x(t)  £  ^{x (t))  (2) 

where  t,  £  R>o  and  'F  :  X  C  Rd  =4  Rd  is  a  set- valued 
map.  A  solution  to  this  dynamical  system  is  an  abso¬ 
lutely  continuous  curve  x  :  [0,  T]  — >  X  which  satisfies  (2) 


almost  everywhere.  The  set  of  equilibria  of  (2)  is  denoted 
by  Eq('F)  =  {x  £  X  |  0  £  ^(a;)}. 

Lemma  2.3  (Existence  of  solutions):  Ford'  :  Rd  =4 

Rd  upper  semicontinuous  with  nonempty,  compact,  and 
convex  values,  there  exists  a  solution  to  (2)  from  any 
initial  condition. 

The  LaSalle  Invariance  Principle  for  set- valued  continuous¬ 
time  systems  is  helpful  to  establish  the  asymptotic 
stability  properties  of  systems  of  the  form  (2).  A  set 
W  C  X  is  weakly  positively  invariant  with  respect  to 
'F  if  for  any  x  £  W,  there  exists  x  £  X  such  that 
x  £  'F(a’).  The  set  W  is  strongly  positively  invariant 
with  respect  to  \F  if  ^(cc)  C  W,  for  all  x  £  W.  Finally, 
the  set-valued  Lie  derivative  of  a  differentiable  function 
V  :  Rd  — >  R  with  respect  to  IF  at  x  £  Rd  is  defined  by 
£^T( x)  =  (u  •  W(x)  |  v  £  ^(2:)}. 

Theorem  2.4  (Set- valued  LaSalle  Invariance 
Principle):  LetW  cXbe  a  strongly  positively  invariant 
under  (2)  and  V  :X-)la  continuously  differentiable 
function.  Suppose  the  evolutions  of  (2)  are  bounded  and 
max£^,T(a:)  <  0  or  L^V{x)  =  0,  for  all  x  £  W.  If 
Sy.v  =  {aa  £  X  |  0  £  Cq,V(x)},  then  any  solution  x(t), 
t  £  R>o,  starting  in  W  converges  to  the  largest  weakly 
positively  invariant  set  M  contained  in  Sq,y  C\W .  When 
M  is  a  finite  collection  of  points,  then  the  limit  of  each 
solution  equals  one  of  them. 


2.3  Graph  theory 

We  present  some  basic  notions  from  algebraic  graph  the¬ 
ory  following  the  exposition  in  (Bullo  et  al.,  2009).  A 
directed  graph,  or  simply  digraph,  is  a  pair  Q  =  (V,£), 
where  V  is  a  finite  set  called  the  vertex  set  and  f  C  VxV 
is  the  edge  set.  A  digraph  is  undirected  if  (v,  u )  £  £  any¬ 
time  (u,v)  £  £.  We  refer  to  an  undirected  digraph  as  a 
graph.  A  path  is  an  ordered  sequence  of  vertices  such  that 
any  ordered  pair  of  vertices  appearing  consecutively  is 
an  edge  of  the  digraph.  A  digraph  is  strongly  connected  if 
there  is  a  path  between  any  pair  of  distinct  vertices.  For 
a  graph,  we  refer  to  this  notion  simply  as  connected.  A 
weighted  digraph  is  a  triplet  Q  =  (V,  £,  A),  where  ( V,£ ) 
is  a  digraph  and  A  £  R>q"  is  the  adjacency  matrix  of  Q, 
with  the  property  that  a,j  >  0  if  (ty,  Vj)  £  £  and  =  0, 
otherwise.  The  weighted  out-degree  and  in-degree  of  V{, 
i  £  {1, ...  ,n},  are  respectively,  d™ut(vi)  —  )Cj=i  ad  an<^ 
dfa (vi)  =  1  aji-  The  weighted  out-degree  matrix  Dout 

is  the  diagonal  matrix  defined  by  (Dout),;i  =  d£ut(i),  for 
all  i  £  {1, . . . ,  n}.  The  Laplacian  matrix  is  L  =  Dout  —  A. 
Note  that  Ll„  =  0.  If  Q  is  strongly  connected,  then  zero 
is  a  simple  eigenvalue  of  L.  Q  is  undirected  if  L  =  LT  and 
weight-balanced  if  d%nt(v)  =  d^(u),  for  all  v  £  V.  Equiv¬ 
alently,  Q  is  weight-balanced  if  and  only  if  l^L  =  0  if  and 
only  if  L  +  LT  is  positive  semidefinite.  Furthermore,  if  Q 
is  weight-balanced  and  strongly  connected,  then  zero  is 
a  simple  eigenvalue  of  L  +  LT.  Note  that  any  undirected 
graph  is  weight-balanced. 
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2-4  Zero-sum  games 

We  recall  basic  game-theoretic  notions  following  Ba§ar 
and  Olsder  (1999).  An  n-player  game  is  a  triplet 
G  =  (P,  X,  t/),  where  P  is  the  set  of  players  with 
|P|  =  n  G  Z>2,  X  =  Xj  x  ...  x  X„,  X;  c  is  the 
set  of  (pure)  strategies  of  player  tq  G  P,  d*  £  Z>1;  and 
U  =  (rti, . . .  ,un),  where  Ui  :  X  — »•  R  is  the  payoff  func¬ 
tion  of  player  iq,  i  G  {1, . . . ,  n}.  The  game  G  is  called  a 
zero-sum  game  if  ui  =  0-  An  outcome  x*  G  X  is 
a  (pure)  Nash  equilibrium  of  G  if  for  all  i  G  {1, . . . ,  n} 
and  all  ^  £  X,, 

Uiix^x*^)  >  ui(xi,x'!Li), 

where  X-i  denotes  the  actions  of  all  players  other  than  iq. 
In  this  paper,  we  focus  on  a  class  of  two-player  zero-sum 
games  which  have  at  least  one  pure  Nash  equilibrium  as 
the  next  result  states. 

Theorem  2.5  (Minmax  theorem):  Let  Xi  C  Rdl 

andX 2  C  ,  d\,d2  G  Z>i,  be  nonempty ,  compact,  and 
convex.  If  u  :  x  X2  — >  K.  is  continuous  and  the  sets 

{xr  G  Xi  |  u(x' ,  y)  >  a }  and  {y'  G  X2  |  u(x,  y')  <  a}  are 
convex  for  all  x  G  Xi ,  y  G  X2,  and  a  G  R,  then 

max  min  u{x ,  y)  =  min  max  u(x,y). 

x  y  y  x 

Theorem  2.5  implies  that  the  game  G  =  ({tq,  tq};  Xi  x 
X2,  (u,  —u))  has  a  pure  Nash  equilibrium. 

3  Problem  statement 

Consider  two  networks  Si  and  E2  composed  of  agents 
{tq , . . . ,  vni }  and  agents  {uq , . . . ,  wn2 },  respectively. 
Throughout  this  paper,  Ei  and  S2  are  either  connected 
undirected  graphs,  c.f.  Section  4,  or  strongly  connected 
weight-balanced  digraphs,  c.f.  Section  5.  Since  the  lat¬ 
ter  case  includes  the  first  one,  throughout  this  section, 
we  assume  the  latter.  The  state  of  S1;  denoted  aq,  be¬ 
longs  to  Xi  c  Rdl,  d\  G  Z>i.  Likewise,  the  state  of  S2, 
denoted  x2,  belongs  to  X2  C  Rd2 ,  d2  G  Z>i.  In  this  pa¬ 
per,  we  do  not  get  into  the  details  of  what  these  states 
represent  (as  a  particular  case,  the  network  state  could 
correspond  to  the  collection  of  the  states  of  agents  in 
it).  In  addition,  each  agent  tq  in  Si  has  an  estimate 
x\  G  Mdl  of  what  the  network  state  is,  which  may  differ 
from  the  actual  value  aq .  Similarly,  each  agent  iVj  in  S2 
has  an  estimate  x2  G  of  what  the  network  state  is. 
Within  each  network,  neighboring  agents  can  share  their 
estimates.  Networks  can  also  obtain  information  about 
each  other.  This  is  modeled  by  means  of  a  bipartite 
directed  graph  Seng,  called  engagement  graph,  with  dis¬ 
joint  vertex  sets  {tq, . . . ,  vni}  and  {uq, . . . ,  wn2},  where 
every  agent  has  at  least  one  out-neighbor.  According  to 
this  model,  an  agent  in  Si  obtains  information  from  its 
out-neighbors  in  Seng  about  their  estimates  of  the  state 
of  S2 ,  and  vice  versa. 

Figure  1  illustrates  this  concept. 


Fig.  1.  Networks  Ei  and  E2  engaged  in  a  strategic  scenario. 
Both  networks  are  strongly  connected  and  weight-balanced, 
with  weights  of  1  on  each  edge.  Edges  which  correspond  to 
Eeng  are  dashed. 

For  each  i  G  {1, . . .  ,rii},  let  f{  :  Xi  x  X2  — >  R  be  a 
locally  Lipschitz  concave-convex  function  only  available 
to  agent  tq  G  Si.  Similarly,  let  f2  :  Xi  x  X2  — >  R  be  a 
locally  Lipschitz  concave-convex  function  only  available 
to  agent  uij  G  S2,  j  G  {1, . . . ,  n2}.  The  networks  Si  and 
S2  are  engaged  in  a  zero-sum  game  with  payoff  function 
U  :  Xi  x  X2  — >  M 

rii  n2 

U{x1,x2)  =  '^/fl(x1,x2)  =  I,x2),  (3) 

*= 1  3— 1 

where  Si  wishes  to  maximize  U,  while  S2  wishes  to  min¬ 
imize  it.  The  objective  of  the  networks  is  therefore  to 
settle  upon  a  Nash  equilibrium,  i.e.,  to  solve  the  follow¬ 
ing  maxmin  problem 

max  min  U( Xi,x2).  (4) 

2?l(zXi  012^X2 

We  refer  to  the  this  zero-sum  game  as  the  2-network  zero- 
sum  game  and  denote  it  by  Gacjv-net  =  (Si,  S2,  Seng,  U). 
We  assume  that  Xi  C  Rdl  and  X2  C  Rd2  are  compact 
convex.  For  convenience,  let  aq  =  (x\, . . . ,  xrf1)T  and 
x2  =  {x2, . . . ,  x22)T  denote  vector  of  agent  estimates 
about  the  state  of  the  respective  networks. 

Remark  3.1  (Power  allocation  in  communication 
channels  in  the  presence  of  adversaries):  Here 
we  present  an  example  from  communications  inspired 
by  (Boyd  and  Vandenberglre,  2004,  Section  5.5.3).  Con¬ 
sider  n  Gaussian  communication  channels,  each  with 
signal  power  jq  G  R>o  and  noise  power  rp  G  R>o,  for 
i  G  {1, ...  ,n}.  The  capacity  of  each  channel  is  propor¬ 
tional  to  log(l  +  /3pi/((Ji  +  77 j)),  where  /3  G  R>o  and 
oq  >  0  is  the  receiver  noise.  Note  that  capacity  is  con¬ 
cave  in  pi  and  convex  in  .  Both  signal  and  noise  powers 
must  satisfy  a  budget  constraint,  i.e.,  Y^i=iPi  =  P  an<^ 
]C"=i  Vi  =  C-  f°r  some  given  P,  C  G  R>o-  Two  networks 
of  n  agents  are  involved  in  this  scenario,  one,  Ei,  select¬ 
ing  signal  powers  to  maximize  capacity,  the  other  one, 
E2,  selecting  noise  powers  to  minimize  it.  The  network 
Ei  has  decided  that  m\  channels  will  have  signal  power 
aq,  while  n  —  1  —  m\  will  have  signal  power  x2-  The  re¬ 
maining  nth  channel  has  its  power  determined  to  satisfy 
the  budget  constraint,  i.e.,  P  —  m\Xi  —  (n  —  1  —  ?ni)a;2. 
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Likewise,  the  network  X2  does  something  similar  with 
m2  channels  with  noise  power  yi,  n  —  1  —  m2  channels 
with  noise  power  y2,  and  one  last  channel  with  noise 
power  C  —  ?tt-2 J/i  —  (n—  1  -777.2)7/2-  Each  network  is  aware 
of  the  partition  made  by  the  other  one.  The  individual 
objective  function  of  the  two  agents  (one  from  Si,  the 
other  from  X2)  making  decisions  on  the  power  levels 
of  the  zth  channel  is  the  channel  capacity  itself.  For 
i  £  {1, . . . ,  n  —  1},  this  takes  the  form 

f(x,y)  =  log(l+-^-), 

'  CTi  +  Vb  > 


for  some  a,b  £  {1,2}.  Here  x  =  (xi,x2)  and  y  =  (2/1 , 2/2) - 
For  i  =  n ,  it  takes  instead  the  form 


fn(x,y)  =  log  (l  + 


(3(P  —  m\X\  —  (n  —  1  —  mi)x2)  \ 

on  +  C  -  m2yi  -  (n  -  1  -  m2)y2' 


Note  that  J}"=r  fl{xiV)  is  the  total  capacity  of  the  n 
communication  channels.  • 

3.1  Reformulation  of  the  2-network  zero-sum  game 

In  this  section,  we  describe  how  agents  in  each  net¬ 
work  use  the  information  obtained  from  their  neighbors 
to  compute  the  value  of  their  own  objective  functions. 
Based  on  these  estimates,  we  introduce  a  reformulation 
of  the  Ga<jv-net  =  (Si,  S2,  Xeng,  U)  which  is  instrumen¬ 
tal  for  establishing  some  of  our  results. 

Each  agent  in  Si  has  a  locally  Lipschitz,  concave-convex 
function  /}  :  Rdl  x  Rd2™2  — »  R  with  the  properties: 
(Extension  of  own  payoff  function):  for  any  X\  £ 
Rdl ,  x2  £  Rd2 , 

f[(xi,ln2®x2)  =  f[{xi,X2).  (5a) 

(Distributed  over  Seng):  there  exists  f\  :  Rdl  x 
j^d2|N2eng(ws)|  jg,  sucj1  that,  for  any  x\  £  Rdl 
x2  £  R^2™2, 

fl(x  i,x2)  =f\(x1,n\(x2)),  (5b) 


with  n\  :  Rd2Tl2  — >  Rda  l-^eng  (*'*) I  the  projection  of  X2 
to  the  values  received  by  zy  from  its  out-neighbors 
in  Seng. 

Equation  (5a)  states  the  fact  that,  when  the  estimates 
of  all  neighbors  of  an  agent  in  the  opponent’s  network 
agree,  its  evaluation  should  coincide  with  this  estimate. 
Equation  (5b)  states  the  fact  that  agents  can  only  use 
the  information  received  from  their  neighbors  in  the  in¬ 
teraction  topology  to  compute  their  new  estimates. 

Each  agent  in  X2  has  a  function  /|  :  Rdini  x  Rda  — >  R 
with  similar  properties.  The  collective  payoff  functions 


of  the  two  networks  are 


n  1 

U1(x1,x2)  =  '^2fl(x\,x2),  (6a) 

i= 1 
n  2 

U2(x1,x2)  =  '^2fZ(x1,xJ2).  (6b) 

3= 1 

In  general,  the  functions  U\  and  U2  need  not  be  the  same. 
However,  U\(lni  ®  X\,  lni  ®  x2)  =  U2(lni  ®  x\,  lni  ® 
x2),  for  any  x\  £  Rdl,  x2  £  Rda-  When  both  functions 
coincide,  the  next  result  shows  that  the  original  game 
can  be  lifted  to  a  (constrained)  zero-sum  game. 
Lemma  3.2  (Reformulation  of  the  2-network 
zero-sum  game):  Assume  that  the  individual  pay¬ 
off  functions  {fi)Zi>  satisfying  (5)  are  such 

that  the  network  payoff  functions  defined  in  (6)  sat¬ 
isfy  U\  =  U2,  and  let  U  denote  this  common  function. 
Then,  the  problem  (4)  on  Rdl  x  Rd2  is  equivalent  to  the 
following  problem  on  R“1'il  x  R™2^2, 

max  min  U{x i,x2), 
x\  ex}1  a,2ex”2 

subject  to  L1x1  =  Onidl,  L2x2  =  0„2d2,  (7) 

with  Li(  =  Li  ®  I  de  and  the  Laplacian  of  He,  (-  £  {1,2}. 

Proof.  The  proof  follows  by  noting  that  (i)  U(  lni  ® 
x\,l „2  ®  x2 )  =  U(x i,x2)  for  all  Xi  £  Rdl  and  x2  £ 
Rda  and  (ii)  since  Q\  and  Q2  are  strongly  connected, 
LlXl  =  0nidl  and  L2£c2  =  0n2d  iff  xx  =  lni  ®  xx  and 
x2  =  ln2  ®  x2  for  some  x\  £  R^  and  x2  €  Rd2.  □ 

Remark  3.3  (Restrictions  on  extensions):  The  as¬ 
sumption  of  Lemma  3.2  does  not  hold  in  general  for  all 
sets  of  extensions  satisfying  (5a)  and  (5b).  If  the  interac¬ 
tion  topology  is  one-to-one  (i.e.,  both  networks  have  the 
same  number  of  agents,  the  interaction  topology  is  undi¬ 
rected,  and  each  agent  in  the  first  network  obtains  infor¬ 
mation  only  from  one  agent  in  the  opposing  network), 
the  natural  extensions  satisfy  the  assumption.  Exam¬ 
ple  5.5  later  provides  yet  another  instance  of  a  different 
nature.  In  general,  determining  if  it  is  always  possible  to 
choose  the  extensions  in  such  a  way  that  the  assumption 
holds  is  an  open  problem.  • 

We  denote  by  Gadv-net  =  (Si,  S2,  Xe„g,  U)  the  con¬ 
strained  zero-sum  game  defined  by  (7)  and  refer  to 
this  situation  by  saying  that  Gadv_net  can  be  lifted 
to  Gadv_net.  Our  objective  is  to  design  a  coordination 
algorithm  that  is  implementable  with  the  information 
that  agents  in  Si  and  S2  possess  and  leads  them  to  find 
a  Nash  equilibrium  of  Gadv-net,  which  corresponds  to 
a  Nash  equilibrium  of  Gadv_net  by  Lemma  3.2.  Achiev¬ 
ing  this  goal,  however,  is  nontrivial  because  individual 
agents,  not  networks  themselves,  are  the  decision  mak¬ 
ers.  From  the  point  of  view  of  agents  in  each  network, 
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the  objective  is  to  agree  on  the  states  of  both  their  own 
network  and  the  other  network,  and  that  the  resulting 
states  correspond  to  a  Nash  equilibrium  of  Ga(jv_net. 

The  function  U  is  locally  Lipschitz  and  concave- 
convex.  Moreover,  from  Lemma  2.2,  the  elements  of 
dXlU(xi,  X2)  are  of  the  form 

9(x  1,®2)  =  ( 9(x{,X2 )’  •  •  •  ’  5(xJ,a!2))  G  dXlU{x\,X2), 


where  g\xi  2)  G  dXlfl(x\,x2),  for  i  G  {1, . . .  ,ni}.  Note 

that,  because  of  (5b),  we  have  dXlf[(x\,  1„2  ®  X2)  = 
axi/|(a;l,a:2).  A  similar  reasoning  can  be  followed  to  de¬ 
scribe  the  elements  of  dX2U(xi,X2).  Next,  we  present  a 
characterization  of  the  Nash  equilibria  of  Gadv-net,  in¬ 
strumental  for  proving  some  of  our  upcoming  results. 
Proposition  3.4  (Characterization  of  the  Nash 
equilibria  of  Gacjv-net):  For  £1,  £2  strongly  connected 
and  weight-balanced,  define  Fj  and  F2  by 


F1(x1,z1,x2)  =  -U(xi,x2)  +  XiL1zi  +  ^xfLisci, 

F2(X2,Z2,X1)  =  U(X1,X2)  +  X2L2Z2  +  ^ X2~L2x2 ■ 

Then,  F\  and  F2  are  convex  in  their  first  argument,  lin¬ 
ear  in  their  second  one,  and  concave  in  their  third  one. 
Moreover,  assume  (x\,  z*,  x\,  z|)  satisfies  the  following 
saddle  property  for  (Fj,  F2):  (x^,  z*)  is  a  saddle  point  of 
(xi,zi)  i->-  F1(xi,z1,X2)  and  (x%,  ztf)  is  a  saddle  point 
of{x2,z2)  F2\x2,Z2,x\).  Then, 

(i)  (cc},  z(  +  lni  ®  cq,  xt,,  Z2  +  1  n2  0  a2 )  satisfies  the 
saddle  propeHy  for  (Fi,  F2)  for  any  a\  G  Rdl ,  a2  G 
Rd2,  and 

(ii)  {x\,x\)  is  a  Nash  equilibrium  of  Gadv-net- 
Furthermore, 

(Hi)  if  (x\,x 2)  is  a  Nash  equilibrium  of  Gadv-net  then 
there  exists  z( ,  such  that  (x } ,  z } ,  x\ ,  zt) )  satisfies 

the  saddle  property  for  (Fi ,  F2)  ■ 


E”=i  g[. 


=  0 


d\ 


and  E"=i  =  °d2’  Le‘ 


(a:},®?;)  is  a  Nash  equilibrium.  Finally  for  proving  (iii), 
note  that  x\  =  lni  ®  x\  and  x\  =  1„2  0  a^.  The  result 
follows  then  from  the  fact  that  0  G  dXlU(x*,X2)  and 
0  G  dX2U(x*,  X2)  implies  that  there  exists  z(  G  R"ldl 
and  G  R"2d2  with  Li z\  G  dXlU(x\,X2)  and  L2ZI  G 
-dX2U(  xt,x*2).  □ 


4  Distributed  convergence  to  Nash  equilibria 
for  undirected  topologies 

In  this  section,  we  introduce  a  distributed  dynamics 
which  solves  (7)  when  £1  and  £2  are  undirected.  In  par¬ 
ticular,  we  design  gradient  dynamics  to  find  points  with 
the  saddle  property  for  ( Fi ,  F 2 )  prescribed  by  Proposi¬ 
tion  3.4.  Consider  the  set-valued  dynamics  ^Nash-midir  : 
(Mrfini)2  x  (Rd2"2)2  (Rdi"i)2  x  (Rd2n2)2, 


x\  +  \j\X\  +  L1Z1  G  dXlU(x\,X2),  (8a) 

zi  =  L1X1,  (8b) 

±2  +  L2X2  +  L2z2  G  -dX2U(xi,X2),  (8c) 

Z2  =  L2x2,  (8d) 


where  xe,Z(  G  Wnede,  £  G  {1,2}.  Note  that  (8a)-(8b) 
and  (8c)-(8d)  correspond  to  saddle-point  dynamics  of 
Fi  in  (x\,Z\)  and  F2  in  (*2,22),  respectively.  Local  so¬ 
lutions  to  this  dynamics  exist  by  virtue  of  Lemmas  2.1 
and  2.3.  We  characterize  next  its  asymptotic  conver¬ 
gence  properties. 

Theorem  4.1  (Distributed  convergence  to  Nash 
equilibria  for  undirected  networks):  Consider  the 
zero-sum  game  Ga(jv.net  =  (£1;  £2,  £eng,  U),  with  £j 
and  £2  connected  undirected  graphs,  Xi  C  Rdl ,  X2  C  Rda 
compact  and  convex,  and  U  :  Xx  x  X2  — >  R  strictly 
concave-convex  and  locally  Lipschitz.  Assume  Gadv-net 
can  be  lifted  to  Gadv-net-  Then,  the  projection  onto  the 
first  and  third  components  of  the  solutions  of  (8)  asymp¬ 
totically  converge  to  agreement  on  the  Nash  equilibrium 
of  G adv-net  ■ 


Proof.  The  statement  (i)  is  immediate.  For  (ii),  since 
(x\ ,  z( ,  x*2 ,  z2 )  satisfies  the  saddle  property,  and  the  net¬ 
works  are  strongly  connected  and  weight-balanced,  we 
have  x\  =  lrai  ®  x\,  x\  G  Rdl,  *2  =  1„2  0^2,  x \  G  Rd2, 
Liz}  G  —daslU(x*,x:2),  and  L2z2  G  dX2U(x\,  x%). 
Thus  there  exist  G  dXlf{(x\,x 2),  i  G 

{l,...,ni},  and  e  -<9x2/2 (z*, ^2)>  3  G 

(1, . . . ,  n2},  such  that 

Li*i  =  (<?i\(x^*p -•••><, (x;,x*)))T>  ancl 
L2^2  =  (ff2,(x*,x*)>  >  ’  •  '  >52,(x*,x*)))T' 

Noting  that,  for  £  G  {1,2},  (1^  ®  =  (1^  ® 

ldJ(L<  c  I de)  =  L  ®  I di  =  Odtxdene,  we  deduce  that 


Proof.  Throughout  this  proof,  since  property  (5b) 
holds,  without  loss  of  generality  and  for  simplicity 
of  notation,  we  assume  that  agents  in  £1  have  ac¬ 
cess  to  x2  and,  similarly,  agents  in  £2  have  access 
to  X\.  By  Theorem  2.5,  a  solution  to  (6)  exists.  By 
the  strict  concavity-convexity  properties,  this  solu¬ 
tion  is,  in  fact,  unique.  Let  us  denote  this  solution  by 
x\  =  lrai®a:}  and®?;  =  l^®^-  By  Proposition  3. 4(iii), 
there  exists  z(  and  z 2  such  that  {x\,  z*,  x*2,  z^f)  G 
Eq( 'pNash-undir ) •  First,  note  that  given  any  initial  con¬ 
dition  (xi,  Zi,X2,  zl))  G  (R”ldl)2  x  (R"2<i2)2,  the  set 

Wz°1,z°  =  {(x1,Z1,X2,Z2)  I 

(In,  ®  I dt)ze  =  (1  Tni  ®  ldt)z°,  £  G  {1,  2}}  (9) 
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is  strongly  positively  invariant  under  (8).  Consider  the 
function  V  :  ( Rdini )2  x  (Rd2Tl2)2  ->  R>0  defined  by 

V(x1,z1,x2,z2) 

=  ^(xi~  xi)T(xi  ~x*i)  +  \(zi-  zi)Tizi  -  zl) 

+  \{x2-  x*2)T(x2  -  x*2)  +  \{z2-  z2)t(z2  -  zl). 

The  function  V  is  smooth.  Next,  we  examine  its 
set-valued  Lie  derivative  along  ’I'Nash-undir-  Let  £  G 
Z'S’NMh-undirV^i’  z!,x2,  z2).  By  definition,  there  exists 
V  G  ^Nash-undir(*i,  Zi,  x2,z2),  given  by 

v  =  (  —  Li^i  —  L 1Z1  +  g  i,(Xl,x2)) 

—  h2x2  —  Tj2z2  —  g2gXl,x2)i  L2x2), 

where  gi,(Xl,Xa)  G  dXlU(x1,x2)  and  52, (c El,®2)  S 
dX2U{x\,x2),  such  that 

£  =  v  ■  VV(xi,z1,x2,z2) 

=  Oi  -  x\ )T(-Lia;i  -  L1Z1  +  ffi,(Xl>*2)) 

+  (cc2  -  xl)T(-L2x2  -  L2z2  -  g2,(Xl,Xa)) 

+  (zi  -  z})T LiXx  +  (z2  -  zl)T L2x2. 


As  a  by-product,  we  conclude  that  the  trajectories 
of  (8)  are  bounded.  By  virtue  of  the  set-valued  version 
of  the  LaSalle  Invariance  Principle,  cf.  Theorem  2.4, 
any  trajectory  of  (8)  starting  from  an  initial  condition 
(a?5,  Zi ,  x2,  z2)  converges  to  the  largest  positively  invari¬ 
ant  set  M  in  S'vtNash_undir,y  C  U_1(<  V{x^,  z°,  x°2l  z°)). 
Let  (x1,z1,x2,z2)  G  M.  Because  M  C  S«'Nash.undir)v, 
then  Fi(x\,  z*,  xi,)  —  F\{x-i,  z\,  x%)  =  0,  i.e., 

— U{x\,  x2)  +  U(x  1,  *2)  —  cc^LiZ*  —  ^a;fLi£Ci  =  0. 

(10) 

Define  now  Gi  :  K”ldl  x  R”ldl  x  K”2*  ->  R  by 
G1(x1,zi,x2)  =  F1(x1,Zi,x2)  -  \x1[L1x1.  Gi  is  con¬ 
vex  in  its  first  argument  and  linear  in  its  second.  Further¬ 
more,  for  fixed  x2,  the  map  {x\,z\)  i-»-  Gi(a:i, Zi, x2) 
has  the  same  saddle  points  as  {x\,  Z\)  i-»-  Fi(x1,  z\,  x2). 
As  a  result,  G\(x\,  z*,  x%)  —  Gi{xi,z{,x%)  <  0,  or 
equivalently,  —U{x\,xl)  +  U(x i,x2)  —  x\L\z\  <  0. 
Combining  this  with  (10),  we  have  that  LiXg  =  0 
and  —U{x\,xl)  +  U{xi,xQ  =  0.  Since  U  is  strictly 
concave  in  its  first  argument  X\  =  x\.  A  similar  ar¬ 
gument  establishes  that  x2  =  x2.  Using  now  the  fact 
that  M  is  weakly  positively  invariant,  one  can  deduce 
that  L gzg  G  —dX(U(xi,x2),  for  £  G  {1,2},  and  thus 
{x1,z1,x2,z2)  G  Eq(^Nash  -undir)*  ^ 


Note  that  —L1x1—L1zi+g1^XliX2')  G  -dXlF1(x1,z1,x2), 
Liaq  G  dZlF1(x1,z1,x2),  -L2x2  -  L2z2  -  g2>(Xl,Xa)  G 
~dXaF2(x1,z2,x2),  and  L2:e2  G  dZ2F2{x2,  z2,  Xg).  Us¬ 
ing  the  first-order  convexity  property  of  F\  and  F2  in 
their  first  two  arguments,  one  gets 

£  <  Fi(x\,z\,x2)  -  Fi(xi, zi,x2)  +  F2(x*2,z2lxi) 

-  F2(x2,z2,x1)  +  F1(x1,z1,x2)  -  F1(x1,zl,x2) 

+  F2(x2,z2,x1)  -  F2(x2,zl,xi). 

Expanding  each  term  and  using  the  fact  that  (x^,  zl,xl,  z%) 
G  Eq(\k Nash-undir )  1  we  simplify  this  inequality  as 

£  <  —U{x\,x 2)  +  U{ x\,xQ  -  z\LiXi 
-  ^aJiLiaq  -  Z2L2CC2  -  ^x2L2x2. 

By  rearranging,  we  thus  have 


5  Distributed  convergence  to  Nash  equilibria 
for  directed  topologies 

Interestingly,  the  saddle-point  dynamics  (8)  fails  to  con¬ 
verge  when  transcribed  to  the  directed  network  setting. 
This  observation  is  a  consequence  of  the  following  result, 
which  studies  the  stability  of  the  linearization  of  the  dy¬ 
namics  (8),  when  the  payoff  functions  have  no  contribu¬ 
tion  to  the  linear  part. 

Lemma  5.1  (Necessary  condition  for  the  conver¬ 
gence  of  (8)  on  digraphs):  Let  Tig  be  strongly  con¬ 
nected  and  f\  =  0,  i  G  {1, . . . ,  ng},  for  £  G  {1,  2}.  Then, 
the  set  of  network  agreement  configurations  Sagree  = 
{(lni  <8>  X!,  lni  <8>  Zl,  1„2  ®  X2,  1  n2  0  Z2)  G  (R"ldl)2  X 
(R112*)2  |  xg,zg  G  M.di,£  G  {1,2}},  is  stable  under  (8) 
iff. ,  for  any  nonzero  eigenvalue  A  of  the  Laplacian  Lg, 
£  G  {1, 2},  one  has  \/3|Im(A)|  <  Re(A). 

Proof.  In  this  case,  (8)  is  linear  with  matrix 


£  <  -F2{x2l  z*2,x\)  -  Fi(xi,  zl,xl). 

Next,  since  F2(x\,  z| ,  x%)  +Fi(a32,  z%,x *)  =  0,  we  have 

£  <  F^xlzlxl)  -  F^xuzlxl) 

+  F2{x*2,zl,x\)  -  F2{x2,z*2,x*1), 

yielding  that  ^  <  0.  As  a  result, 

maxZ$Nash 

-undir  V{xi,zi,  x2,  z2)  <  0. 


0  \ 

V  0  (fV)®W 


(ii) 


and  has  iSagree  as  equilibria.  The  eigenvalues  of  (11)  are 
of  the  form  \g  (^  ±  ^i) ,  with  X(  an  eigenvalue  of  Lg,  for 
£  G  {1,2}  (since  the  eigenvalues  of  a  Kronecker  product 
are  the  product  of  the  eigenvalues  of  the  corresponding 
matrices).  Since  L g  =  L g  ®  \,i,,  each  eigenvalue  of  ~Lg  is 
an  eigenvalue  of  L g.  The  result  follows  by  noting  that 
Re(A^  ±  'f  i))  =  |(TV^Im(A/)  -  Re(A^)).  □ 
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It  is  not  difficult  to  construct  examples  of  strictly 
concave-convex  functions  that  have  zero  contribution  to 
the  linearization  of  (8)  around  the  solution.  Therefore, 
such  systems  cannot  be  convergent  if  they  fail  the  neces¬ 
sary  condition  identified  in  Lemma  5.1.  The  counterex¬ 
ample  provided  in  our  recent  paper  (Gharesifard  and 
Cortes,  2012c)  of  strongly  connected,  weight-balanced 
digraphs  that  do  not  meet  the  stability  criterium  of 
Lemma  5.1  is  therefore  valid  in  this  context  too. 

From  here  on,  we  assume  that  the  payoff  functions  are 
differentiable.  We  elaborate  on  the  reasons  for  this  as¬ 
sumption  in  Remark  5.3  later.  Motivated  by  the  obser¬ 
vation  made  in  Lemma  5.1,  we  introduce  a  parameter 
a  G  R>o  in  the  dynamics  of  (8)  as 


Xx  +  aLiXi  ^'LiZi  =  'S7U{xi,x2),  (12a) 

z1=L1x1,  (12b) 

x2-\-aL2x2+'L2sz2  = -N7U{x1,x2),  (12c) 

z2  =  L2x2.  (12d) 


and  x2  =  1  n2  ®  x2.  By  Proposition  3.4(i),  there  exists 
(xl,zl,x2,z2)  G  Eq(4,Nash-dir)  n  Wz o)Zo.  Consider  the 
function  V  :  (Rdini)2  x  (Rd2n2)2  ->  R>0, 

V(x1,z1,x2,z2) 

=  \{xi  -  x\)T{xi  -  x\)  +  ^(x2  -  x2)T{x2  -  x2 

+  9  ,zi)  ~  V(x*,z*))  (y(x1,z1)  ~  2/(x*,z*)), 

+  y{y(x2,z2)  y(x*,z|))  (y(x2,z2)  y(x*,z*))> 

where  y(xe,ze)  =  f3x(  +  ze,  £  G  {1,2},  and  (3  G 
R>o  satisfies  f32  —  a(3  +  2  =  0.  This  function  is 
quadratic,  hence  smooth.  Next,  we  consider  £  = 

A-Nash-dir^tei)  2i>*2,  z2)  given  by 

£  =  {—aL\X\  -  L1Z1  +  '\7U(x\,x2),'LiX\,—a'L2x2 
-  h2z2  -  X7U(x  1,x2),L2x2)  ■  W (xi,zi,  x2,  z2). 


We  next  show  that  a  suitable  choice  of  a  makes  the 
dynamics  convergent  to  the  Nash  equilibrium. 
Theorem  5.2  (Distributed  convergence  to  Nash 
equilibria  for  directed  networks):  Consider  the  zero- 
sum  game  Ga<jv-net  =  (Si,  S2,  Seng,  U),  with  Si  and  S2 
strongly  connected  and  weight-balanced  digraphs,  Xi  C 
Rdl ,  X2  C  Rd2  compact  and  convex,  and  U  :  Xi  x  X2  — > 
R  strictly  concave- convex  and  differentiable  with  glob¬ 
ally  Lipschitz  gradient.  Assume  Gadv-net  can  be  lifted  to 
Gadv-net  such  that  U  is  differentiable  and  has  a  globally 
Lipschitz  gradient.  Define  h  :  R>q  — >  M  by 


Hr)  fn 


({( 


r4  +  3 r2  +  2  \  2  ,  r4 

-  —4 

r  / 


+  3r2  +  2 ' 
r  / 


Kr2 


(1  +  r2)  ’ 


(13) 


where  A™m  =  min^=12{A*(L£  +  L J)},  A*(-)  denotes  the 
smallest  non-zero  eigenvalue  and  K  G  R>o  is  the  Lip¬ 
schitz  constant  of  the  gradient  of  U.  Then  there  exists 
(3*  G  R>o  with  h(p*)  =  0  such  that  for  all  0  <  (3  <  [3*, 
the  projection  onto  the  first  and  third  components  of  the 
solutions  of  (12)  with  a  =  ^  £ 2  asymptotically  converge 
to  agreement  on  the  Nash  equilibrium  of  Gadv-net  • 


Proof.  Similarly  to  the  proof  of  Theorem  4.1,  we  as¬ 
sume,  without  loss  of  generality,  that  agents  in  £i 
have  access  to  x2  and  agents  in  S2  to  x\.  For  conve¬ 
nience,  we  denote  the  dynamics  described  in  (12)  by 

^Nash-dir  :  (R6*1”1)2  X  (R^™2)2  (M<hm)2  x  (Rd2n 2)2_ 

Let  (x°,  z°,  x2,  z2)  be  an  arbitrary  initial  condition. 
Note  that  the  set  Wzozo  defined  by  (9)  is  invariant  un¬ 
der  the  evolutions  of  (12).  By  an  argument  similar  to  the 
one  in  the  proof  of  Theorem  4.1,  there  exists  a  unique 
solution  to  (7),  which  we  denote  by  x\  =  lni  ®  x\ 


After  some  manipulation,  one  can  show  that 

2  1 

£  'y  '  2^Xi  ~  x^'  y(xe’ze)  ~  3l(x*,z*))  Affxi,  y(xttXe)) 

e=i  z 

2  x 

+  Eo^’  y?xe,ze))AJ He  -  xhy{*e,ze)  -  »(«»•,»•)) 
1=1  z 
2 

+  ^2,{-l)°~l (xi  -  Xp)T\7xeU(xi,  x2) 
e=i 
2 

T  'y  '(  i)J  P(y(xt,zt)  y(x*,z*))  ^xeU(xi,x2), 

i=i 

where  A(,  £  G  {1, 2},  is 


Af={  -(a-fi)Le  -I*\ 

V(-/3(a  -  ff)  +  1)L/  -PL t)  ‘ 

This  equation  can  be  written  as 
2  1 

£  —  'y  'j  9  He  —  xe ,  y(xe,zg)  ~  y(x*,z*)) 
t=i 

QeHe  xt ,  y(xe,zt)  i/(x*,z*)) 

2 

+  TXxt  —  xti  y(xe,ze)  ~~  y{x*,z*))  A({xt,y^3 5*,z*)) 
1=1 
2 

+  E^(~l y~1He  -  xe)TV xeUHh  x2) 

e=i 

2 

~b  'y  '(  1  Y  Piy{xg,ze)  y(a;*,z*))  ^  xe  U  (x  \ ,  X2) , 

1=1 


where  Qg,  £  £  {1,2},  is  given  by 

Qe  =  (Le  +  Lj)®  •  (14) 

Note  that,  we  have 

^4l  (*®1 ,  V(x*  ,z*  )  )  (Li2/(o,*  ,z*)  , ,z*)) 

=  -(yxiU(x*1,x2),!3VXlU(x*1,X2)), 
^2  (*^2  ,  V(x2  >z2  )  )  (L2?/(a,2  ,z2  )  5  ^H22/(a3*  ,s2  )) 

Thus,  after  substituting  for  y(xe,ze),  we  have 


where  K  £  R>o  is  the  Lipschitz  constant  of  VC7.  We 
thus  conclude  that 

2  1 

£  <  ^2  ^(xg  -  x*e,zt  -  z\)TQg(xg  -  x},zg  -  z\) 

i=  1 

~  {1+jf2)  (\\^7XlU(x1,x2)  ~VXlU(xlx2)\\2 
+  \\VX2U(xi,x2)  -  VX2U(x !,x*2)\\2) 

+  /3(zi  -  zl)T(\7XlU(xi,x2)  -  VXlU(xl,x2)) 

-  (3(z2  -  zl)T{\7X2U(xllx2)  -  Va,217(a;i,  a;^))- 

One  can  write  this  inequality  as  displayed  in  (17),  where 

X  =  (xi  -  x\,Zi  -  zl,x 2  -  X2,z2  -  Z2, 
XXlU(x  1,0:2)  -  VXl17(a:},  x2), 

U(x1,x2)  - 

^  X2  U(xi,x*2)). 


£  =  -  z\)TQg(xg  -  x*e,zg  -  Zg) 

t=i 

+  (1  +/32)(cci  -  ®5|)T(VXlt/(*i,*2)  -  V^t^a:},^)) 

-  (1  +/32)(a;2  -  xl)T{VxJj(xux2)  -  VX2U(x1,  x2)) 
+  ^(z1  -  zl)T (XXlU(x1,x2)  -  XXlU(xl,x2)) 

-  P(Z2  -  z*f(XX2U(x  1,*2)  -  XX2U(Xl,x*2)),  (15) 


where 


<5^ 


}-/33-(^)-/5-(l  +  /32)\ 

(  -(1  +  /S2)  - 13  ) 


(Lg  +  Lj), 


Since  (x1,  zi,  x2,  z2)  £  Wzo<z 0,  we  have  (1^  <8>  ldJ(z*  - 
z\)  =  0de,  £  &  (1, 2},  and  hence  it  is  enough  to  establish 
that  Q  is  negative  semidefinite  on  the  subspace  W  = 

{(vi,V2,V3,V4,V5,V6)  £  (Rnidl)2  X  (R112^2)2  X  Rnidl  X 

R™2d2  |  (J_T  0  \djV2  =  odl,(l£a  ®  ldj>4  =  0„2}.  Using 
the  fact  that  —  -^(1+ P2)\nede  is  invertible,  for  £  £  {1,  2}, 
we  can  express  Q  as 

Q  = 

(Q  1  0  0  0  ^ 

0  02  0  0  „ 

TV  ^  Nt, 

0  0  -i(l  +  /l2)l nidl  0 

v0  0  0  -i(l  +  p2)\n2d2j 

v  x 

D 


for  £  £  {1,  2}.  Each  eigenvalue  of  Qg  is  of  the  form 


where  Qg 


K02 

(1+/32) 


0 


Tilde 


£  £  (1,  2},  and 


„  .  —  (/34  +  3/32  +  2)  ±  \/(/34  +  3/32  +  2)2  -  4/32 

m  =  A, - ^ - , 

(16) 


where  is  an  eigenvalue  of  Lg  +  L J,  £  £  {1,  2}.  Using 
now  Theorem  A.l  twice,  one  for  (xg,x2),  {x*llx2),  and 
another  one  for  (xi,x2),  ( x\,x2 ),  we  have 


(*1  -  x\)T  (X  XlU  (x\,  *2)  -  XXlU(xl,x2))  < 

-  ^  (HVXlU(xlfx2)  -  XXlU{x{,x 2)||2)  , 
(x2  -  x*2)t{VX2U{xi1x2)  -  VX2U(xi,x*2))  < 

-  ^  (ilVX2U(Xl,x2)  -  XX2U(xux*2)\\2)  , 


nidi 

0 

0 

0 

0 

0  > 

0 

\n\d\  0 

0 

&K  1  w 
1+^32  'nidi 

0 

N  = 

0 

0 

0 

0 

ln2d2 

0 

0 

1 n2d2 

0 

0 

0 

0K  1  n 
l_|_/(32  ln2d2 

0 

0 

0 

0 

Inidi 

0 

V  0 

0 

0 

0 

0 

\n2d2  ) 

Noting  that  W  is 

invariant  under  NT  (i.e. , 

NTW  =  W), 

all  we 

need  to  check  is 

that  the  matrix  D  is  negative 

semidehnite  on  W.  Clearly, 


~^(l+02)\„ 


0 

-i(l+/32)l„ 
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£  <  - X 7 
s  -  2 


f Qi,u  Q  1,12  o 
<3i  ,21  <3  i  ,22  0 

o  o  Q2,n 

0  <32,21 


0 
0 

V  o 


0 

0 

<32,12 

<32,22 

0 


0 


0  I3\n2d2 


0 

0 

0 

(1+/32)  I 

if  'nidi 

0 


0 

0 

0 

0 

.Ml 


X, 


K 


^n2d2 / 


Q 


(17) 


is  negative  definite.  On  the  other  hand,  for  £  G  {1,2}, 
on  0  is  an  eigenvalue  of  <3^  with  multiplic¬ 

ity  2d(  and  eigenspace  generated  by  vectors  of  the  form 
(ln/  0  a,  0)  and  (0,  lne  8  b ),  with  a,  b  €  RdG  However, 
on  {(vi,v2)  G  (Rn<W)2  |  (1^  ®  \de)v2  =  Ode},  0  is  an 

eigenvalue  of  Qe  with  multiplicity  de  and  eigenspace  gen¬ 
erated  by  vectors  of  the  form  (lne  8  a,  0).  Moreover,  on 
{(«i,U2)  €  (R"«*)2  |  (1^  ®  \dt)v2  =  0dJ,  the  eigen¬ 
values  of  (1^2)  ( 0  i„°d< )  are  (/^2)  with  multiplicity 

nede  —  di  and  0  with  multiplicity  nede-  Therefore,  us¬ 
ing  Weyl’s  theorem  (Horn  and  Johnson,  1985,  Theorem 
4.3.7),  we  deduce  that  the  nonzero  eigenvalues  of  the 

—  ~  tv''  o  2 

sum  Qe  are  upper  bounded  by  A *(Qe)  +  ej+p2 y-  Thus, 
the  eigenvalues  of  Q  =  (  ^  q2  )  are  upper  bounded  by 

min^=ii2{A*(<3f)}  +  (f+pz)  •  From  (16)  and  the  defini¬ 
tion  of  h  in  (13),  we  conclude  that  the  nonzero  eigenval¬ 
ues  of  <3  are  upper  bounded  by  h(/3).  It  remains  to  show 
that  there  exists  fi*  G  R>o  with  h(/3*)  =  0  such  that 
for  all  0  <  0  <  (3*  we  have  h(0)  <  0.  For  r  >  0  small 
enough,  h{r)  <  0,  since  h(r)  =  —  |A™lnr  +  0(r2).  Fur¬ 
thermore,  lim^oo  h(r)  =  K  >  0.  Hence,  using  the  Mean 
Value  Theorem,  we  deduce  the  existence  of  /?*.  There¬ 
fore  we  conclude  that  £^Nash_dirU(xi,  z\,  X2,  Z2)  <  0. 
As  a  by-product,  the  trajectories  of  (8)  are  bounded. 
Consequently,  all  assumptions  of  the  LaSalle  Invariance 
Principle,  cf.  Theorem  2.4,  are  satisfied.  This  result  then 
implies  that  any  trajectory  of  (8)  starting  from  an  ini¬ 
tial  condition  (x®,  z®,  x®,  z$)  converges  to  the  largest 
invariant  set  M  in  SWNash_dir,v  fl  Wzo  zo.  Note  that  if 
(x1,z1,x2,z2)  G  S*Na.sh_dir,v  n  WzO'Zo,  then  NTX  £ 

ker(<3)  x  {0}.  From  the  discussion  above,  we  know  ker(<3) 
is  generated  by  vectors  of  the  form  (lni  0  ai,  0,0,0), 
(0, 0, 1  n2  8  a2, 0),  ae  G  Rd<!,  j  G  {1, 2},  and  hence  xe  = 
xe  +  1  nt  8  ae,  ze  =  z\.  Using  the  strict  concavity- 
convexity,  this  then  implies  that  xe  =  x\.  Finally,  for 
(a:*,  z\,  X2,  z2)  G  M,  using  the  positive  invariance  of  M, 
one  deduces  that  (**,  Z\,  x\,  z2)  G  Eq(TNash_dir).  □ 

Remark  5.3  (Assumptions  on  payoff  function): 

Two  observations  are  in  order  regarding  the  assump¬ 
tions  in  Theorem  5.2  on  the  payoff  function.  First,  the 


assumption  that  the  payoff  function  has  a  globally  Lip- 
schitz  gradient  is  not  too  restrictive  given  that,  since 
the  state  spaces  are  compact,  standard  boundedness 
conditions  on  the  gradient  imply  the  globally  Lips- 
chitz  condition.  Second,  we  restrict  our  attention  to 
differentiable  payoff  functions  because  locally  Lipschitz 
functions  with  globally  Lipschitz  generalized  gradients 
are  in  fact  differentiable,  see  (Gharesifard  and  Cortes, 
2012c,  Proposition  A.l).  • 

Remark  5.4  (Comparison  with  best-response 
dynamics):  Using  the  gradient  flow  has  the  advan¬ 
tage  of  avoiding  the  cumbersome  computation  of  the 
best-response  map.  This,  however,  does  not  come  for 
free.  There  are  concave-convex  functions  for  which  the 
(distributed)  gradient  flow  dynamics,  unlike  the  best- 
response  dynamics,  fails  to  converge  to  the  saddle  point, 
see  (Feijer  and  Paganini,  2010)  for  an  example.  • 

We  finish  this  section  with  an  example. 

Example  5.5  (Distributed  adversarial  selection 
of  signal  and  noise  power  via  (12)):  Recall  the  com¬ 
munication  scenario  described  in  Remark  3.1.  Consider 
5  channels,  {chi,  ch2,  CI13,  chi,  chs},  for  which  the  net¬ 
work  Ei  has  decided  that  {chi,  ch3}  have  signal  power 
x\  and  {ch2,chi}  have  signal  power  x2-  Channel  ch.5 
has  its  signal  power  determined  to  satisfy  the  budget 
constraint  P  G  R>o,  he.,  P  —  2x\  —  2x2.  Similarly, 
the  network  E2  has  decided  that  chi  has  noise  power 
yi,  {ch2,  ch3,  chi}  have  noise  power  y2,  and  chs  has 
noise  power  C  —  y\  —  3y2  to  meet  the  budget  con¬ 
straint  C  G  R>o-  We  let  x  =  (x1,  x2,  x3,  x4,  x5)  and 
y  =  (2/\y2,y3.,y4,y5),  where  x*  =  (x\,xi)  G  [0,P]2 
and  yl  =  (y\ ,  y2)  G  [0,  C ]2,  for  each  i  G  {1, . . . ,  5}. 

The  networks  Ei  and  E2,  which  are  weight-balanced  and 
strongly  connected,  and  the  engagement  topology  Eeng 
are  shown  in  Figure  2.  Note  that,  according  to  this  topol¬ 
ogy,  each  agent  can  observe  the  power  employed  by  its 
adversary  in  its  channel  and,  additionally,  the  agents  in 
channel  2  can  obtain  information  about  the  estimates 
of  the  opponent  in  channel  4  and  vice  versa.  The  payoff 
functions  of  the  agents  are  given  in  Remark  3.1,  where 
for  simplicity  we  take  cq  =  o\,  for  i  G  {1,3,5},  and 
cq  =  (J2,  for  i  G  {2, 4},  with  <ji,  a2  G  R>o- 
This  example  fits  into  the  approach  described  in  Sec¬ 
tion  3.1  by  considering  the  following  extended  payoff 
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Fig.  2.  Networks  Ei,  E2  and  Eeng  for  the  case  study  of  Ex¬ 
ample  5.5.  Edges  which  correspond  to  Seng  are  dashed.  For 
i€  {!,...  ,5},  agents  Vi  and  Wi  are  placed  in  channel  ch;. 
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Note  that  these  functions  are  strictly  concave  and  thus 
the  zero-sum  game  defined  has  a  unique  saddle  point 
on  the  set  [0,P]2  x  [0,  C]2.  These  functions  satisfy  (5) 
and  Ui  =  U2.  Figure  3  shows  the  convergence  of  the 
dynamics  (12)  to  the  Nash  equilibrium  of  the  resulting 
2-network  zero-sum  game.  • 

6  Conclusions  and  future  work 

We  have  considered  a  class  of  strategic  scenarios  in  which 
two  networks  of  agents  are  involved  in  a  zero-sum  game. 
The  networks  aim  to  either  maximize  or  minimize  a 
common  objective  function.  Individual  agents  collabo¬ 
rate  with  neighbors  in  their  respective  network  and  have 
partial  knowledge  of  the  state  of  the  agents  in  the  other 
one.  We  have  introduced  two  aggregate  objective  func¬ 
tions,  one  per  network,  identified  a  family  of  points  with 
a  special  saddle  property  for  this  pair  of  functions,  and 
established  their  correspondence  between  the  Nash  equi¬ 
libria  of  the  overall  game.  When  the  individual  networks 
are  undirected,  we  have  proposed  a  distributed  saddle- 
point  dynamics  that  is  implementable  by  each  network 


via  local  interactions.  We  have  shown  that,  for  a  class 
of  strictly  concave-convex  and  locally  Lipschitz  objec¬ 
tive  functions,  the  proposed  dynamics  is  guaranteed  to 
converge  to  the  Nash  equilibrium.  We  have  also  shown 
that  this  saddle-point  dynamics  fails  to  converge  for  di¬ 
rected  networks,  even  when  they  are  strongly  connected 
and  weight-balanced.  Motivated  by  this  fact,  we  have  in¬ 
troduced  a  generalization  that  incorporates  a  design  pa¬ 
rameter.  We  have  shown  that  this  dynamics  converges 
to  the  Nash  equilibrium  for  strictly  concave-convex  and 
differentiable  objective  functions  with  globally  Lipschitz 
gradients  for  appropriate  parameter  choices.  An  inter¬ 
esting  venue  of  research  is  determining  whether  it  is  al¬ 
ways  possible  to  choose  the  extensions  of  the  individual 
payoff  functions  in  such  a  way  that  the  lifted  objective 
functions  coincide.  Future  work  will  also  include  relax¬ 
ing  the  assumptions  of  strict  concavity-convexity  and 
differentiability  of  the  payoff  functions,  and  the  glob¬ 
ally  Lipschitz  condition  on  their  gradients,  extending 
our  results  to  dynamic  interaction  topologies  and  non¬ 
zero  sum  games,  and  exploring  the  application  to  various 
areas,  including  collaborative  resource  allocation  in  the 
presence  of  adversaries,  strategic  social  networks,  collec¬ 
tive  bargaining,  and  collaborative  pursuit-evasion. 
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A  Appendix 

The  following  result  can  be  understood  as  a  generaliza¬ 
tion  of  the  characterization  of  cocoercivity  of  concave 
functions  (Golshtein  and  Tretyakov,  1996,  Lemma  6.7). 
Theorem  A.l  (Concave-convex  differentiable 
functions  with  globally  Lipschitz  gradients):  Let 
f  :  Rdl  x  Rd2  — >  R  be  a  concave-convex  differentiable 
function  with  globally  Lipschitz  gradient  (with  Lipschitz 
constant  K  £  R>0y).  For  (x,y),  {x' ,y')  £  Rdl  x  Rda, 

(x  -  x')T(\/xf{x,y)  -  Vxfix',  y')) 

+  (y~  y')T(Vyf{x',  y')  -  Wyf{x,  y)) 

<-^(\\Vxf(x,y')-\7xf(x',y')\\2 

+  \\Vvf(x',y)-Vyf(x',y')\\2 
+  \\\7xf(x',y)~\7xf(x,y)\\2 

+  \\X7yf(x,y/)~X7yf(x,y)\\2). 

Proof.  We  start  by  noting  that,  for  a  concave  function 
j  :  Rd  — >  R  with  globally  Lipschitz  gradient,  the  fol¬ 
lowing  inequality  holds,  see  (Golshtein  and  Tretyakov, 
1996,  Equation  6.64), 

j(x)  <j*  -  ^:||Vi(a:)||2,  (A.l) 

where  j*  =  sup^gRd  j(x)  and  M  is  the  Lipschitz  constant 

of  V).  Given  (x'.y')  £  Rdl  x  Rd2,  define  the  map  f  : 
rA  x  R^2  — y  R  by 

/( x,  y)  =  f{x,  y)  -  f(x',  y)-(x-  x')TVxfix',  y) 

+  f{x,  y)  -  f{x ,  y')  +  {y-  y')TVyf(x,  y'). 

Since  the  gradient  of  /  is  Lipschitz,  the  function  /  is 
differentiable  almost  everywhere.  Thus,  almost  every¬ 
where,  we  have 

Va;/(x,  y)  =  VJ(i,  y)  -  Vxf(x',  y)  +  Vxf(x,  y) 

-  Vxf(x,y')  -{y-  y')TX/ xX/ yf(x,  y'), 
v yf(x ,  y)  =  Vyf(x,  y)  -  Vyf(x',  y)  +  Vyf(x ,  y) 

-  S7yf(x,  y')  -(x-  x')TVyVxf(x',y). 

In  particular,  note  that  Vxf(x',y')  =  Vyf(x',y')  =  0. 
Since  x  H >  f(x,  y')  and  y  H >  fix' ,  y)  are  concave  and  con¬ 
vex  functions,  respectively,  we  can  use  (A.l)  to  deduce 

f(x,y')  <  Vxf(x,y’)  -  Vxfix',y') ||2,  (A.2a) 

~f(x',y)  <  --U|  Vyf(x',y)  -  Vyf(x',y')\\2,  (A.2b) 


where  we  have  used  the  fact  that  supxeRdi  fix,  y')  = 
infyeR^  fix y)  =  fix' ,  y')  =  0.  Next,  by  definition  of /, 

fix ,  y')  =  fix ,  y')  -  fix' ,  y')  -  (x  -  x')TVxf(x',  y'), 
fix',  y)  =  fix',  y)  -  fix',  y')  -  (y  -  y')TS7vfix',  y'). 

Using  (A. 2),  we  deduce  that 
f(x,y')  -  fix' ,y) 

~{x~  x')TS7xfix',  y')  +  (y-  y')TVyfix',  y') 

<  -^(\\Vxfix,y')  -Vxf(x',y')\\2 

+  \\Vvf(x',y)-Vyf(x',y')\\2).  (A.3) 

The  claim  now  follows  by  adding  together  (A.3)  and 
the  inequality  that  results  by  interchanging  (x,y)  and 
ix',  y')  in  (A.3).  □ 
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